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Title: The Impact of a Research-Based Intervention on the Proportional Reasoning of 
Seventh-Grade Students with Mathematics Difficulties: A Regression Discontinuity 
Analysis 


Background and Focus of Study: 

Ratio and proportional relationships, along with the interrelated topics of fractions, 
decimals, and percent, provide a critical foundation for algebra (National Mathematics Advisory 
Panel, 2008). Proportional reasoning, which not only requires understanding the concept of ratios 
and that two or more ratios are equal, also requires the ability to extract relevant information to 
develop a representation of the problem situation and is challenging for many children and 
adolescents, especially students with mathematics difficulties (Ozgiin-Koca & Altay, 2009). 

The intervention investigated in this study represents an approach to proportional problem 
solving via schema-based instruction (SBI). Jitendra, Harwell, Dupuis, and Karl (2017a) 
examined the effectiveness of SBI for a subsample of students with mathematics difficulties 
(MD) selected from a randomized cluster study (Jitendra et al., 2015). Based on a sample of 806 
students classified as MD clustered within 82 classrooms, Jitendra et al. (2017a) reported SBI on 
average improved student scores on a posttest and delayed posttest of proportional problem 
solving (PPS) administered 9 weeks after treatment compared to a control condition, suggesting 
SBI could be used effectively for students classified as having MD in the short and longer term. 
This finding raises an important question: What is the range of proportional reasoning skills and 
general mathematical proficiencies for which SBI enhances student performance? The present 
study used a regression discontinuity approach to identify the boundaries of the effectiveness of 
SBI for students with and without MD. 


Setting and Population: 

Our sample of 1,492 seventh-grade students in 36 schools located in two U.S. states was 
taken from Jitendra, Harwell, Im, Karl, and Slater (2017b). Student and school characteristics 
such as the percentage of students eligible for a free/reduced price lunch, percentage of English 
language learners, and the percentage of Black students were generally lower than national 
averages (U.S. Department of Education, 2015). 


Intervention: 
A detailed description of SBI can be found in Jitendra et al. (2015). 


Research Design and Analysis: 

The multilevel model of Jitendra et al. (2017b) could be used to predict outcome values 
for different combinations of covariates but the many possible combinations prompted us to seek 
an alternative method. We employed a regression discontinuity (RD) because it provides direct 
information about the effectiveness of SBI and because of its ability to support strong inferences 
(What Works Clearinghouse, 2014). The logic of RD is typically based on the crucial role of 
pretests in taking student differences into account (Steiner, Cook, Shadish, & Clark, 2010), in 
that students with similar pretest scores can often be treated as approximately equal on 
background variables, enhancing causal inferences (Bloom, 2010). Similar to Jitendra et al. 
(2017a) GMADE pretest scores below that corresponding to the 35" percentile (10 or less) led to 
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a student being categorized as having MD and above the 35th percentile as non-MD with the 
GMADE posttest as the outcome; unlike Jitendra et al. (2017a) we used the PPS pretest score 
corresponding to the 35" percentile as the MD cutoff score (9 or less) for the PPS posttest and 
delayed posttest outcomes and included both MD and non-MD students in the analyses. The 
analyses compared SBI and control students using normal-theory-based multiple regression in 
which pretest with a selected range of scores defined the sample and treatment (SBI = 1, Control 
= 0) served as a covariate, with PPS posttest, PPS delayed posttest, and the GMADE posttest 
serving as outcomes. Each outcome was analyzed separately using the SPSS 22.0 software 
package (IBM Corp., 2015). 


Results: 

Student and teacher demographic characteristics in the SBI and control conditions are 
presented in Table 1. Following the example of Robinson (2010) we conducted two-sample ¢- 
tests to learn whether SBI and control conditions produced similar outcomes for students with 
PPS and GMADE pretest scores right at the cutoff of MD status. Table 2 shows statistically 
significant differences between SBI and control conditions for students whose PPS pretest score 
was 9 or 10 on the PPS posttest ((252) = 5.068, p < .001, d = 0.63SD) and delayed posttest 
(t(243) = 3.340, p =.001, d= 0.43SD) with SBI students outperforming control students. For the 
GMADE posttest there was a statistically significant difference between SBI and control 
conditions for students whose GMADE pretest score was 10 or 11 (#(235) = 2.746, p = .006, d= 
0.36SD), with SBI students outperforming control students. These findings suggest that the 
impact of the treatment on all three outcomes was similar for students categorized as having (or 
almost having) MD. 

The RD results are reported in Table 3 and Figure 1 and are based on different 
bandwidths of the PPS and GMADE pretests. There was a significant effect of SBI on PPS 
posttest and delayed posttest scores for every bandwidth studied. For example, the bandwidth of 
+ 1 only used students whose PPS pretest score was 8, 9, or 10 (7 = 376) and found a significant 
treatment effect (slope) of 2.11 which is the estimated discontinuity. That is, for students with 
PPS pretest scores of 8, 9, or 10 SBI students on average scored 2.11 points higher on the PPS 
posttest than control students (p < .001, d= 0.54SD); in Figure | this is represented by the 
discontinuity between the two bolded lines for the PPS pretest score of 9. Similar results 
appeared for the GMADE posttest. The fact there were significant treatment effects for every 
bandwidth studied for each outcome suggests the SBI intervention is effective for a relatively 
broad range of proportional reasoning skills and general mathematical proficiencies. 


Conclusion 

Using a regression discontinuity approach provided evidence SBI is effective for a wide 
range of proportional reasoning skills and general mathematical proficiencies and thus broadens 
the population of students SBI can be used effectively with. Standardized slopes for the 
treatment effect also provide evidence the magnitude of the SBI effect is non-negligible, 
although the effect was strongest for the PPS posttest and weaker for PPS delayed posttest and 
the GMADE posttest. 
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Table 1. Summary of Student Demographic Information+ 


Treatment 
SBI Control Total 
n % n % n % 
Student Information 
Age M (SD) 12.60 0.6 12.55 04 12.57 0.4 
Sex Female 268 50.5 283 = 48.0 551 49.2 
Male 254 47.8 300 =—.50.9 554 49.5 
Missing (age and sex) 9 1.7 6 1.0 i 1.3 
Race White 318 59.9 281 = 47.7 599 53.5 
Hispanic 116 =. 21.8 196 = 33.3 312 21.9 
Black 49 9.2 53 9 102 9.1 
Asian 21 4 34 5.8 55 4.9 
Multiracial 18 3.4 19 3.2 37 33 
ELL Yes 46 8.7 69 11.7 115 10.3 
No 476 = 89.6 514 87.3 990 88.4 
SpEd Yes 51 9.6 59 10.0 110 9.8 
No 471 88.7 524 = 89.0 995 88.8 
Missing 9 1.7 6 1.0 15 t3 
FRL Yes 134 825.2 159 = 27.0 293 26.2 
No 33 6.2 85 14.4 118 10.5 
Missing 364 ~=—- 68.6 345 = 558.6 709 63.3 
Teacher Information 
Sex Female 31 91.2 18 72.0 49 83.1 
Male 3 8.8 uf 28.0 10 16.9 
Math courses taken M (SD) rise) 4.8 11.12 10.3 9.05 ve 
EAUCAOR COUN iS) 3.85 3.8 5 10.8 434 75 
taken 
YGASERECHENCE ID. «75D 10.76 7.0 9.84 10.5 10.4 8.6 
math 
PD hours in math M (SD) 19.47 = 13.3 2A. 23:5 20.3 18.1 


Note. + is defined by students whose GMADE pretest scores were 4-14. FRL = students eligible 
for free or reduced priced lunch; ELL = English language learner; SpEd = students qualified for 
special education services. 
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Figure 1. The effect of SBI on PPS posttest (left), delayed posttest (middle), and GMADE 
posttest scores (right) by PPS or GMADE pretest scores. The vertical distance between the solid 
lines as they approach the threshold (i.e., vertical “jump” in outcome variables) is the regression- 
discontinuity-based effect estimate. The gray shading represents the 95% confidence interval 
around the line of best fit. 
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